Lyrics readability vs the English proficiency

Countries of study participants

Introduction

Songs that we listen to every day do not only stimulate our mood, and help as get through our daily chores. They tell us stories, to which we can relate, describe events that we dream of, or simply circulate around words that define our life routines. The lyrics play an integral part in our daily musical sphere. While there is no doubt that the character of the lyrics differs between artists, as well as genres, a question emerges, whether their sophistication has a certain correlation on the “other” side. Namely, if the listeners’ own musical sophistication, and other characteristics (such as background, English proficiency) are connected to the character of the lyrics of the songs they listen to most often. The studies on the musical and language sophistication have not yet been developed extensively, with most of the research focusing on more general relations within the area of music culture - e.g. the one between the music listening and cultural adaptation in host states (provide citation). Thus, our research had as its objective the provision of a valuable point for discussion.

Methodology

The research was based on a survey, to which we collected 55 complete responses. While the number of the total participants was almost twice as high, lack of response to one of the questions already precluded them from being taken into account. To gather the data that would allow us to measure numerous variables connected to the musical sophistication and language, we have asked the participants of our survey some general questions concerning their personal information - their age group, native language, English proficiency, as well as home country. Further, they were asked about their musical sophistication, with the use of inquiries selected from Gold MSI. Additionally, we asked each participant to share a link to their Spotify On Repeat playlist. In order to study the data, we have used Spotify and Genius APIs, and a number of indices for the measurement of the readability of the lyrics - Flesch Reading Ease, Flesch Kincaid Grade Level, Gunning Fog Index, Smog Index, Automated Readability Index, Coleman Liau Index, Linsear Write, Dale Chall Readability formula, and McAlpine EFLAW Readability Score. Due to the limitations of these indices, the only songs we could measure were English-language songs.

Discussion and conclusions

While we have managed to gather responses from people of various nationalities, the majority of the respondents were coming from the age group 18-23 (over 80%). Further, their level of English proficiency was also largely limited to “advanced” and “native”, with none of the participants claiming a “low” level of proficiency.

As regards the second part of the survey, the participants were characterized by a wide range of musical sophistication. While the lowest score amounted to 39, the highest was 115.

Nevertheless, regardless of these above mentioned differences, the outcome of our research provided, that there was little, if any correlation between the character of the lyrics of the songs we listen to every day, and the musical sophistication of the listener. The results that were the closest to showing a certain level of positive relation between these two categories related to the sophistication score, and the average sentiment of the lyrics (correlation of 0.162). However, this correlation needs to be approached critically, as the sample group cannot be regarded as representative of a bigger population, with the p-value of 0.24.

The readability metrics have also been compared to the declared English proficiency of the participants. Likewise, it can be said that there appears to be no relation between the readability score and the English proficiency. The box plots of the main page could suggest that listeners with moderate proficiency are listening to less readable lyrics, however this result is likely the outcome of a low number of data points in this group (5).

The results of the research indicate, that contrary to a belief that one might have, musical sophistication, as well as English language proficiency do not clearly correlate with the character of the lyrics of the (English) songs we listen to. Nevertheless, more extensive and longer research, especially one that would include songs with non-English lyrics, might provide shifts in these scores, and show that in some cases the correlation is more apparent.

Article review

“Data Science Approach to Compare the Lyrics of Popular Music Artists” Caleb Rosebaugh, L. Shamir, 2022

Summary

The article “Data Science Approach to Compare the Lyrics of Popular Music Artists” by Rosebaugh and Samir presents a toolset for quantitative analysis of song lyrics and applies this data science-based approach to explore stylistic differences of chosen artists. The authors highlight that the trends in lyrics evolve over time and are influenced by manifold factors, such as the political environment and genre. Analyzing those trends with the tools provided by the digitization of the contemporary era is important, as music has a large effect on our day-to-day lives.

To demonstrate the tools, Rosebaugh and Samir use a dataset comprising 18,577 songs from 89 influential artists from the 1960s to 1990s. The lyrics are sourced from AZlyrics.com. The choice of artists was based on sufficient data availability. For the study the UDAT tool was employed, which extracts various lyric descriptors, namely: readability indices, sentiments, use of punctuation characters, word and sound diversity, parts of speech frequency and topic frequency. With those variables, the Weighted Nearest Distance (WND) algorithm is used for artist classification, with a focus on extracting knowledge, rather than achieving a reliable classification.

The results of the WND model reveal a 12.3% accuracy in associating songs with the correct artists, which surpasses the random guess accuracy of 1.1%, indicating a potential capture of unique patterns associated with individual artists. Analysing the variables that constitute this result, the variation in readability indices is notable. Artists like Bob Dylan exhibit higher readability and e.g. Guns N’ Roses display lower readibility, suggesting noticeable distinctions in lyrical complexity with the use of the chosen indices. The sentiment analysis set used identifies differences among artists, with Ringo Starr and George Harrison expressing more positive sentiments, contrasting with the negativity found in Hip-Hop artists like Gil Scott-Haron and Rock bands like Motorhead. The correlation between readability and sentiment suggests that more negative songs often utilize longer and more complex language.

Exploration of sound diversity using the Soundex algorithm highlights artists like Imagine Dragons and Bon Jovi using repetitive sounds, while Pink Floyd opts for a more varied set. The related word diversity shows Elton John and Pink Floyd’s lower repetition compared to Imagine Dragons, suggesting a simpler, joyful style versus a high diversity of impactful lyrics. The additional analysis of gendered lyrics reveals variations in the frequency of terms related to men and women, with ZZ Top and The Beatles mentioning women-related terms more frequently.

Overall, the metrics used in the paper seem to successfully describe part of the styles of the popular music lyrics. Their significance is that they are quantitative, making them intuitive and easily explainable, compared to some other “black box” descriptors often used in the field. The analysis included shows how the described metrics could be used in further, more specified research. The Udat software developed for this study was also made freely available and enables easier exploration of quantitative aspects in music lyrics, offering a valuable tool for diverse research questions in the field.

Evaluation

The article of Rosebaugh and Shamir serves as an excellent starting point for future research using the analysis of lyrics. It exemplifies how the Universal Data Analysis of Text tool (UDAT) (Shamir, 2020) could be applied to the music use case. The quantitative data the methodology provides also gives it a broader application, as the findings allow trying to emulate an artist’s style in a more informed way, which is harder with the ‘back box’ approach. However, the tool has a major limitation of supporting only English text, mainly American English. As the authors explore how UDAT can be used in future research, this is a significant piece of information, which should be included in the article. For my application, such support would be necessary, which means that modification of the open-sourced code will be required.

The choice of artists included in the study is also described very vaguely, undermining the scientific value of the observations, which serve more as demonstrations and curiosities, rather than actionable insights. The authors don’t test any hypothesis, but perform data exploration. This is important to remember when looking at the results and is not highlighted in the text, making it not immediately obvious. Perhaps this is also the reason why the authors don’t visualize the interactions between different calculated descriptors in any way, although some statistics on them are provided (like the correlation between readability and sentiment). Such charts would definitely enhance the importance of the insights of the analysis.

Lastly, the limitations of the sampling methodology of the Weighted Nearest Distance model predictions should be discussed in the paper. The result of 12.3% looks promising when compared to the random accuracy, however, it is influenced by many factors, which could highly skew it. The multiclass classification implemented has a lot of artists to choose from for each track, hence its accuracy may be highly dependent on the variance of the features in the chosen samples. The authors ensure that the classes are balanced by taking only 100 songs from each artist, however, they don’t explain how the songs were selected, which could highly affect the accuracy, considering that the artist’s style may change over time.

In conclusion, Rosebaugh and Shamir’s article lays a good foundation for future research in lyric analysis and demonstrates the viability of the metrics proposed by the UDAT tool in the music context. While the lack of a scientific objective in the analysis makes its conclusions scarcely applicable, it still acts as an example use of data science techniques for other endeavours. However, before using these methodologies, all researchers should consider their limitations, which are scarcely described in the article.

References

Rosebaugh, Caleb, and Lior Shamir. “Data Science Approach to Compare the Lyrics of Popular Music Artists.” Unisia, 3 July 2022, pp. 1–26, https://doi.org/10.20885/unisia.vol40.iss1.art1. Accessed 14 Dec. 2022.

Shamir, Lior. “UDAT: Compound Quantitative Analysis of Text Using Machine Learning.” Digital Scholarship in the Humanities, vol. 36, no. 1, 13 Mar. 2020, pp. 187–208, https://doi.org/10.1093/llc/fqaa007. Accessed 11 Nov. 2022.

Further reading

Rosebaugh, Caleb, and Lior Shamir. “Data Science Approach to Compare the Lyrics of Popular Music Artists.” Unisia, 3 July 2022, pp. 1–26, https://doi.org/10.20885/unisia.vol40.iss1.art1. Accessed 14 Dec. 2022.

Shamir, Lior. “UDAT: Compound Quantitative Analysis of Text Using Machine Learning.” Digital Scholarship in the Humanities, vol. 36, no. 1, 13 Mar. 2020, pp. 187–208, https://doi.org/10.1093/llc/fqaa007. Accessed 11 Nov. 2022.

Müllensiefen D, Gingras B, Musil J, Stewart L (2014) “The Musicality of Non-Musicians: An Index for Assessing Musical Sophistication in the General Population.” PLoS ONE 9(2): e89642. https://doi.org/10.1371/journal.pone.0089642

Baker, D. J., Ventura, J., Calamia, M., Shanahan, D., & Elliott, E. M. (2020). “Examining musical sophistication: A replication and theoretical commentary on the Goldsmiths Musical Sophistication Index.” Musicae Scientiae, 24(4), 411-429. https://doi.org/10.1177/1029864918811879

Slevc, L. R., & Miyake, A. (2006). “Individual Differences in Second-Language Proficiency: Does Musical Ability Matter? Psychological Science”, 17(8), 675-681. https://doi.org/10.1111/j.1467-9280.2006.01765.x

Lipovetsky, S. “Readability Indices Structure and Optimal Features.” Axioms 2023, 12, 421. https://doi.org/10.3390/ axioms12050421

Baker, David John ; Ventura, Juan ; Calamia, Matthew ; Shanahan, Daniel ; Elliott, Emily M. „Examining musical sophistication: A replication and theoretical commentary on the Goldsmiths Musical Sophistication Index.”Musicae scientiae, 2020, Vol.24 (4), p.411-429.

Overall

Number

Countries

Languages

Average Gold

Languages in the study

Frequency ranking of artists included in the study

Frequency ranking of all artists in the playlists

Correlations of chosen metrics

Word frequency in the songs included in the project (Top 500, no stopwords)

Top 500 words (no stopwords)